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Abstract 

A model-based, distributed architecture integrates diverse components in a system 
designed for lunar and planetary surface operations: spacesuit biosensors, cameras, 
GPS, and a robotic assistant. The system transmits data and assists communication 
between the extra-vehicular activity (EVA) astronauts, the crew in a local habitat, and 
a remote mission support team. Software processes (“agents”), implemented in a 
system called Brahms, run on multiple, mobile platforms, including the spacesuit 
backpacks, all-terrain vehicles, and robot. These “mobile agents” interpret and 
transform available data to help people and robotic systems coordinate their actions 
to make operations more safe and efficient. Different types of agents relate platforms 
to each other (“proxy agents”), devices to software (“comm agents”), and people to 
the system (“personal agents”). A state-of-the-art spoken dialogue interface enables 
people to communicate with their personal agents, supporting a speech-driven 
navigation and scheduling tool, field observation record, and rover command system. 

An important aspect of the engineering methodology involves first simulating the 
entire hardware and software system in Brahms, and then configuring the agents into 
a runtime system. Design of mobile agent functionality has been based on 
ethnographic observation of scientists working in Mars analog settings in the High 
Canadian Arctic on Devon Island and the southeast Utah desert (Clancey 2002a). The 
Mobile Agents system is developed iteratively in the context of use, with people 
doing authentic work. This paper provides a brief introduction to the architecture and 
emphasizes the method of empirical requirements analysis, through which 
observation, modeling, design, and testing are integrated in simulated EVA 
operations. 

Project Background 

The Mobile Agents project anticipates exploration of Mars, in which a crew of six 
people live in a habitat for many months. One long-term objective is to automate the 
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role of CapCom in Apollo, in which a person on Earth (in Houston) monitored and 
managed the navigation, schedule, and data collection during lunar traverses 
(Clancey, in press b). Because of the communication time delay this function cannot 
be performed from Earth during Mars exploration, and other crew members will often 
be too busy with maintenance, scientific analysis, or reporting to attend to every 
second of a four to seven hour Extra- Vehicular Activity (EVA) (Clancey in press a). 

We have previously described how the Brahms simulation system (Clancey et al. 
1998; Sierhuis 2001) has been adapted to provide both a tool for specifying 
multiagent systems and an implementation architecture for runtime agent interactions 
on mobile platforms (Clancey, et al. 2003). We have described how empirical studies 
at analog work sites, combined with studies of Apollo lunar traverses, provide 
automation ideas and metrics for developing tools and protocols for Mars exploration 
(Clancey 2002a; in press b). Previous publications have outlined two preliminary 
field tests at Johnson Space Center and Meteor Crater in 2002. We have emphasized 
that building a practical system in a difficult terrain prioritizes issues of network 
robustness and diminishes, at least initially, theoretical questions about agent 
competitiveness and cooperation. 

The design of the Mobile Agents system has several theoretical dimensions. The 
overall design hypothesis is that automation can make EVAs more efficient and safe, 
with less human supervision than on Apollo. Partly automation is required by 
diminished mission support in a Mars mission configuration; partly it is enabled by 
digital media and telemetry (e.g., photos don’t need to be physically returned on 
film). The evolving system is based on and contributes to theories of collaboration 
and dialogue management, particularly during field science and mission operations 
(Clancey in press b). Brahms itself is based on a theory of activities, that is, norm- 
based practices by which tasks are actually accomplished (Clancey et al. 1998, 
2002b). Finally, part of our research is the formalization of an engineering 
methodology that deliberately integrates historical and baseline studies, analog 
(simulation) expeditions, modeling, and iterative prototyping in authentic work 
settings. 

Mobile Agents April 2003 Test Configuration 

Mobile Agents is a distributed, multiple-agent system (cf. Pynadath & Tambe 2003; 
Sycara, et al. 2003) in which agents are software programs in the Brahms activity- 
based language (Clancey 2002b). The agents contain and are linked by models that 
make “intelligent” operation possible. The agents are “mobile” because they are on 
movable platforms (backpacks, ATVs, robots). The models in the Mobile Agent 
architecture include: 

• Agents representing people in the simulation system (used for testing the design 
protocols in pre-field test workflow simulations) 

• Models of devices (e.g., camera) used in simulations 

• Dynamic location model, including each agent and object (in terms of “areas” 
such as a habitat, and then specified by the Lat/Long coordination system) 

• Network connectivity model, distributed in design of Comm and Proxy agents 
(respectively, relating external devices and agents to a local platform) 
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• EVA Activity Plan: Sequence of activities specifying start and stop coordinate 
locations, duration, and thresholds allowed. 

» Language model (Table 1): Word models and mapping of phrases to commands 
(with agent, location, object parameters) 

% Command semantics, distributed in agent interactions — a work flow for 
communicating, accessing, and storing data (e.g., photographs, biosensors) 

• Alarms, representing data thresholds (e.g., expected length of an activity within 
an EVA) and actions (e.g., whereto communicate this information). 

• ERA plans, locations, and control procedures (e.g., take photograph of location) 

In preparation for the April 2003 test, the project team developed Brahms systems 

to run respectively on laptops located: 1) on the EVA Robotic Assistant (“ERA”; 
Shillcutt et al. 2002), 2) on spacesuit backpacks (“AstroBrahms”), and 3) in the Mars 
Desert Research Station habitat (MDRS hab) near Hanksville, UT (“HabCom”). The 
ERA model controls the ERA through voice commands (Figure 1). The two 
AstroBrahms systems provide astronaut data and voice command links to the rest of 
the system. The HabCom model, like the person it assists, monitors the EVA activity 
and biosensors worn by the astronauts. Platforms are connected on a wireless 
network (MEX-Alena et al. 2004; KAoS-Bradshaw, et al. 1997). The six biosensors 
are wired to an iPaq PDA worn by the astronaut, then transmitted by bluetooth to a 
Minibook computer attached to the top of backpack. A GPS unit, camera, and 
headphone-microphone are all connected to the Minibook. The ERA Brahms 
controlled the ERA’s camera through an API to the ERA’S computer. Language was 
developed using the REALIST dialog system for each function to support natural 
spoken phrasings by the astronauts during the EVA (Table 1). 

Based on previous experience in the Arizona monsoon, we chose to work in MDRS 
to have a permanent field shelter for working in April’s rainy, cold, windy conditions 
(also sometimes dry and hot with dust storms). Satellite services from NASA/Glenn 
and NREN (Ames) provided an internet protocol telephone (NetPhone) to allow 
communications between support personnel at an EVA site and HabCom in MDRS. 

The astronaut-geologists were provided with scripts to indicate the sequence of 
activities, including locations, and requested or optional commands to test. The 
astronauts could skip or repeat activities by indicating what they were doing (subject 
to predefined location and timing constraints). This was most useful for the authentic 
exploration at Lith Canyon, where Astro2 improvised a voice annotation (Table 2; 
Key: Activity Duration, Duration threshold in minutes, Location threshold in meters.) 

The communication between people, software, and devices is quite complex 
(Figure 1): Astronauts and HabCom communicate via radio and NetPhone relay; 
people speak to their Personal Agents (PAs) on local computers using a microphone 
and receive feedback on a headset. PAs are communicating locally with external 
systems via “communications agents,” providing an Application ‘Programming 
Interface (API) to read data and control devices (e.g., camera). PAs (all implemented 
in Brahms on different computers) communicate via a wireless network (with 
repeaters) using the KAoS “agent registration” system. 
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Table 1: Functionality tested April 2003 with representative voice commands 


Mobile Agents Functionality 

Example Voice Command 

Location Tracking 

• GPS mounted on backpacks & ATVs 

• Flexible logging interval 

• Sent to Remote Science Team (RST) 
at variable intervals 

• Naming (predetermined vocabulary) 

start tracking my location every <N> 
seconds 

call this location <location name> 
where is [<location name> | Boudreaux]? 

Science Data Logging 

• Sample bags associated with 
locations 

• Voice annotations associated with 
sample bags and locations 

• Photographs logged by time & place 

• Stored at hab & transmitted to RST 

create sample bag # 
associate {this} [voice note|sample bag 
{#}] with docation name> 
record/play {a} voice note {associated 
with docation name> | sample bag #} 
upload [all | one] image {s} 

Biosensor Logging 

• Transmitted via iPaq to backpack 

• Logged at intervals & interpreted 

start tracking my biosensors every 
<N> seconds 

Activity Tracking 

• Indicate start from predetermined list 

• Model of location, duration, sequence 

• Alerting for exceeding thresholds 

start <activity name> activity 
how much time is left {until next 
activity}? 

ERA Commanding 

• Move, follow, take a picture 

Boudreaux take a picture of [tracked 
astronaut | docation name>] 


Table 2: Lith Canyon Scenario script (first half) provided to Astronauts. 

ACTIVITY PLAN I GEOLOGISTS’ SCRIPT ~ 

<EVA PREP> 1 . Drive in EVEREST with backpacks, helmets, suits, 

all equipment 

2. Start Minibooks & GPS 

3 . Don suit with boots, gaiters, radios & headsets 

4. Put on backpack & helmet & connect cables 

Checking equipment 1. “Start CHECKING EQUIPMENT activity” 

(20, 5, 20) 2. “Start tracking my location every 60 seconds” 

3. “Start tracking my biosensors every 5 minutes” 

4. “Start WALKING TO TOP OF CANYON activity” 

Walking to top of canyon {Astronaut 2 improvised a voice note during the walk} 

(10, 0, 10) 

Sample fossils 1. “Start SAMPLE FOSSILS activity” 

(10, 5, 0) 2. Sample bag, voice annotation, association, photo 

3 . “Start WALK TO HEAD OF CANYON activity” 

Walk to head of canyon < Walk carefully down the hill and proceed to the head of 
(1 0, 0, 1 0) the canyon to the south (your left) > 
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Figure 1 . Simplified schematic of MA configuration April 2003 
Scenarios and Results 

Six test scenarios were designed, ranging from a simple walk around the hab to a full- 
day drive onto a plateau. Actual testing involved incremental, pre-planned stages: 

1) Wired test in the lower deck of MDRS to confirm communications protocols and 
peripheral connections 

• 2) Wireless test inside the hab (without full suits, emphasizing communications and 
biosensors) 

3) Test standing on the front porch of MDRS (allows use of GPS for first time) 

4) Full walk (over one hour) around MDRS, following a script to test basic 
functionality; all systems running except ERA. 

5) “Pedestrian EVA” with ERA, walking from porch to a dry wash about 100 meters 
south, gathering samples, taking photos, commanding ERA to take a photo, and 
return 

6) “Lith Canyon EVA” involved two repeaters, ATV providing gateway to a LAN in 
the canyon, and a hour or more of scripted sampling and photography. 

After two days of setup at and around MDRS, the scenarios were accomplished 
very gradually. Nearly 5 of 1 1 workdays were devoted to model modifications and 
testing inside MDRS. Functionality errors were discovered from incomplete end-to- 
end simulation (caused by inadequate resources and planning). Repeatedly, we 
discovered complex interactions, such that every scenario constituted a new test, 
making perfection unimportant at each stage. For example, the GPS unit wouldn’t 
work inside, and originally every scenario was based on a GPS signal being received 
before the first voice co mm and was given. Not unexpectedly, network multi-pathing 
in the hills around MDRS caused erratic connections and high-volume data 
forwarding and alarms bogged down the network. Some missing capabilities that 


5 
























might have seemed advanced were actually key for testing; for example, starting the 
agent system required every platform to be running, complicating incremental tests. 

Figure 2 shows the topography configuration for Lith Canyon; it required nearly 
two days to deploy communication equipment in preparation for this EVA. The Lith 
Canyon site involves broken ledges and steep cliffs 5 km from MDRS. The 
topography caused several serious problems: 

• A wireless “shadow” occurred at the head of the canyon (as expected), causing 
the computerized backpacks to drop out of the network linking the astronauts 
back to MDRS. (Communications were properly queued and handled when 
possible.) 

• The astronauts were unwilling to pass over a sharp, meter-high drop off in the 
canyon, requiring them to change the plan and walk around. 

• The ERA was unable to follow the astronauts into the canyon because of the 
terrain, and even along the ledge had to be directly teleoperated with a person 
standing nearby. 



Figure 2. Topographic layout of MDRS (hab), Cisco repeaters, ATV LAN gateway, 
and astronauts with ERA in Lith Canyon (about 5 km from MDRS). 


The Lith Canyon field test was a major accomplishment for the Mobile Agents 
project. The geologists’ backpack computers running Brah m s were wirelessly 
networked to another computer on an ATV 75 m away on a ledge across the canyon, 
and from there to a laptop running in MDRS more than 5 km away. The EVA lasted 
about 1.5 hours, as batteries allowed. 


Conclusions 

Field test results can be summarized according to lessons learned about the hardware, 
the agent architecture, and logistics of setting up the system and carrying out the 
scenarios: 

■ Hardware 

— Technology required for field science is strongly topography driven 

— A robot should be capable of working in terrain that geologists explore 

— Need automated antenna and video tracking to monitor the astronauts 

— Adapt ERA’s differential GPS for astronauts 

— Biosensors require more robust connectivity to backpack computers 
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■ Agent Architecture 

— Copes well with loss of signal; but Astro PAs must take over some HabCom 
monitoring functions when nodes become isolated 
— HabCom needs automated map to locate people, robot, ATVs and alarm log 
— Need assertions to verify end-to-end functionality (e.g., RST received photo?) 
— Voice feedback to positively confirm every command (e.g., “Do you want to 
start Checking Equipment activity?”) and to indicate the status of processing 
was developed during the field modifications and proved highly beneficial 
— Must integrate speech output, astronaut voice, and HabCom for recording 
— Astronauts work in parallel, lack voice loop; they must coordinate to avoid 
work redundancy (e.g., need ability to record conversation as a “voice note”) 

■ Logistics 

— Need formal data network specification (GPS, computer, radio, biosensors) 

— Need written specs/deliverables (not just design documents) 

— Need field backups for all computers 

Ideally, mobile agents should run for the duration of an entire mission — perhaps 
several years. However, agent interactions with the environment result in an ever- 
expanding memory of beliefs. This can result in serious performance degradation of 
performance because of increased time in evaluating conditional actions. An interim 
solution, implemented during the field test, is to retract transitory communication 
beliefs (i.e., that a request has been received). A more systematic, built-in model of 
forgetting is required. 

We learned a great deal about voice feedback, scenario design, network robustness, 
and system monitoring, exemplifying our approach of “empirical requirements 
analysis.” One may design clever agent algorithms and architectures, but in practice 
simple services are needed that are never considered back in the home lab. In 
particular, we are frequently discovering new tools required to monitor and verify the 
system’s operation during this developmental process. This phase can be expected to 
last many years, and involves viewing the agents as assisting in the research and 
ongoing redesign of the system. 

As suggested by the previous tests, we conclude that a multiagent simulation with 
scenario-based formal specification accelerates cross-institution collaboration for 
integrating sensors, automation, procedures, and communications. We plan to return 
to MDRS in April 2004 to complete the scenarios with extended navigation and 
schedule monitoring, a medical agent to interpret bio-signs, augmented tools for the 
HabCom person to monitor the EVA, and a science database shared with the RST. 
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